Remote administration of BICAMS measures and the Trail-Making Test to assess cognitive impairment in multiple sclerosis.

Reliable remote cognitive testing could provide a safer assessment of cognitive impairment in multiple sclerosis (MS) during the COVID-19 pandemic and thereafter. Here we aimed to investigate the reliability and feasibility of administering Brief International Cognitive Assessment for MS (BICAMS) and the Trail-Making Test (TMT) to people with MS online. Between-group differences on BICAMS and the TMT were examined in a sample of 68 participants. Group 1 (N = 34) was tested in-person pre-pandemic. Group 2 was tested remotely. Within-group differences for in-person and virtual administrations were examined for Group 1. No significant differences between virtual and in-person administrations of the CVLT-II and SDMT were detected. BVMT-R scores were significantly higher for virtual administrations (M = 20.59, SD = 6.65) compared to in-person administrations (M = 16.35, SD = 6.05), possibly indicating inter-rater differences. Strong positive correlations were found for in-person and virtual scores within Group 1 on the CVLT-II (r = .84), SDMT (r = .85), TMT-A (r = .88), TMT-B (r = .76) and BVMT-R (r = .72). No significant differences between in-person and remote administrations of CVLT-II and SDMT in people living with MS were detected. Recommendations for future studies employing the TMT and BVMT-R online are provided.

From face-to-face to remote evaluation of teacher-education candidates during the COVID-19 pandemic

Although teacher's personality is an essential component of successful classroom learning, most teacher education programs accept students solely on the basis of scholastic ability scores such as school grades, national test scores (SAT, GRE) or undergraduate college transcripts. To ensure suitability to teaching, a personality-evaluation system was developed in Israel for teacher education candidates. This evaluation system includes non-cognitive measures, such as group dynamic exercises, simulations, a teaching exercise, situational judgement tests, personality tests and an inter-personal interview, all performed face-to-face (FTF) at a testing center. The outbreak of COVID-19 in 2020 brought about lockdowns and social distancing, precluding the administration of this FTF evaluation system. Therefore, the development team adapted the system to online remote testing, using Zoom technology. The present study examined the effect of this transition to remote evaluation on the quality of selection for teaching, looking at suitability-to-teaching scores and the subjective views of applicants and evaluators. A comparison of the 2020 remote scores with the 2019 FTF scores revealed that scores on remote evaluation were slightly lower than FTF scores, and were more centralized. While the candidates found that remote evaluation provided fewer opportunities to express themselves, both candidates and evaluators were satisfied with the administration and convenience of the evaluation day. The Discussion chapter summarizes the unique affordances and constraints of remote evaluations and presents suggestions for changes which might be made when moving an assessment online that could take advantage of this new environment.

USABILITY TESTING OF INTERACTIVE SYSTEMS CONDUCTED BY UNIVERSITY STUDENTS DURING A PANDEMIC

This case study describes a usability testing course in which students learn by practicing several evaluation methods. The on-campus format makes it possible for teachers and students to meet to discuss recorded test sessions and students can observe other students' execution of pilot studies conducted on campus. The COVID-19 pandemic placed new demands on this course. In-person activities were avoided by some students and many test participants. Some student teams tried remote usability testing. Interestingly, screen recordings (with sound) of the test sessions show that remote testing sometimes helped the students focus more on observation and less on (inappropriately) guiding the test subjects. Another effect was that the students found it easier to recruit participants than during the previous years when the university was teeming with students, lecturers, and non-academic staff. However, the recruited participants were often notably limited to the students' circles of friends. © (2022) by Association for Information Systems (AIS) All rights reserved.

Passive Video Quality Assessment Using Lab and Remote Testing

In this chapter, research about the assessment of video quality for gaming content will be provided. At first, a dataset that was used for the development of the ITU-T Rec. G.1072 will be presented. The dataset was created in a laboratory environment using the passive test paradigm described in Chap. 3. Next, some results of the collected video quality ratings will be illustrated. While QoE assessment studies traditionally make use of controlled laboratory environments, there are also other possibilities to conduct user studies without a laboratory environment. Especially during the COVID-19 pandemic, which prevented many researchers from performing lab studies, the concept of supervised and unsupervised remote studies got lots of attention. By using such a remote study design, two studies assessing video quality ratings of similar conditions as in the previously mentioned dataset were conducted. These two studies allow to address three research topics that will be the focus of the remainder of this chapter. At first, it will be investigated whether video quality ratings obtained using the remote study design are comparable to those collected in the lab environment. Second, a comparison between video quality ratings collected using a stimulus duration of 20 s instead of 30 s will be performed, which tries to answer whether it is enough to use a shorter stimulus duration as proposed in ITU-T Rec. P.809. Lastly, the differences between using a discrete 5-point ACR scale and the extended continuous 7-point scales will be investigated. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

An Investigation Into the Impact of Test Session Disruptions for At-Home Test Administrations

In response to the closures of test centers worldwide due to the COVID-19 pandemic, several testing programs offered large-scale standardized assessments to examinees remotely. However, due to the varying quality of the performance of personal devices and internet connections, more at-home examinees likely suffered "disruptions” or an interruption in the connectivity to their testing session compared to typical test-center administrations. Disruptions have the potential to adversely affect examinees and lead to fairness or validity issues. The goal of this study was to investigate the extent to which disruptions impacted performance of at-home examinees using data from a large-scale admissions test. Specifically, the study involved comparing the average test scores of the disrupted examinees with those of the non-disrupted examinees after weighting the non-disrupted examinees to resemble the disrupted examinees along baseline characteristics. The results show that disruptions had a small negative impact on test scores on average. However, there was little difference in performance between the disrupted and non-disrupted examinees after removing records of the disrupted examinees who were unable to complete the test. © The Author(s) 2022.

Remote videolink observation of model home sampling and home testing devices to simplify usability studies for point-of-care diagnostics [version 1;peer review: awaiting peer review]

Both home sample collection and home testing using rapid point-of-care diagnostic devices can offer benefits over attending a clinic/hospital to be tested by a healthcare professional. Usability is critical to ensure that in-home sampling or testing by untrained users does not compromise analytical performance. Usability studies can be laborious and rely on participants attending a research location or a researcher visiting homes;neither has been appropriate during COVID-19 outbreak control restrictions. We therefore developed a remote research usability methodology using videolink observation of home users. This avoids infection risks from home visits and ensures the participant follows the test protocol in their home environment. In this feasibility study, volunteers were provided with models of home blood testing and home blood sampling kits including a model lancet, sampling devices for dried blood spot collection, and model lateral flow device. After refining the study protocol through an initial pilot (n = 7), we compared instructions provided either as written instructions (n = 5), vs addition of video instructions (n = 5), vs written and video instructions plus videolink supervision by the researcher (n = 5). All users were observed via video call to define which test elements could be assessed remotely. All 22 participants in the study accessed the video call and configured their videolink allowing the researcher to clearly observe all testing tasks. The video call allowed the researcher to assess distinct errors during use including quantitative (volume of blood) and qualitative (inaccurate interpretation of results) errors many of which could compromise test accuracy. All participants completed the tasks and returned images of their completed tests (22/22) and most returned completed questionnaires (20/22). We suggest this remote observation via videolink methodology is a simple, rapid and powerful methodology to assess and optimise usability of point-of-care testing methods in the home setting.

Untying the educational evaluator’s hands: a preliminary tele-assessment framework

Despite knowledge of COVID-19’s expected impact on the 2020 and 2021 academic school years, policymakers, professional organizations, and test publishers have failed to offer consistent, well-defined or corresponding advice to educational evaluators on how to meet the unique challenges the pandemic has introduced. The directive vacuum that was produced led individuals and school districts to create their own policies and practices as a temporary stopgap. As a consequence, diagnosticians and evaluators across the country were left to interpret confusing legal requirements and competing professional recommendations concerning whether students could be evaluated remotely, whereby social distancing could be observed while continuing to address the ever-present academic needs of our youth. This essay calls for the development of clear testing standards for remote testing. [ FROM AUTHOR] Copyright of Preventing School Failure is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full . (Copyright applies to all s.)

Passive Video Quality Assessment Using Lab and Remote Testing

In this chapter, research about the assessment of video quality for gaming content will be provided. At first, a dataset that was used for the development of the ITU-T Rec. G.1072 will be presented. The dataset was created in a laboratory environment using the passive test paradigm described in Chap. 3. Next, some results of the collected video quality ratings will be illustrated. While QoE assessment studies traditionally make use of controlled laboratory environments, there are also other possibilities to conduct user studies without a laboratory environment. Especially during the COVID-19 pandemic, which prevented many researchers from performing lab studies, the concept of supervised and unsupervised remote studies got lots of attention. By using such a remote study design, two studies assessing video quality ratings of similar conditions as in the previously mentioned dataset were conducted. These two studies allow to address three research topics that will be the focus of the remainder of this chapter. At first, it will be investigated whether video quality ratings obtained using the remote study design are comparable to those collected in the lab environment. Second, a comparison between video quality ratings collected using a stimulus duration of 20 s instead of 30 s will be performed, which tries to answer whether it is enough to use a shorter stimulus duration as proposed in ITU-T Rec. P.809. Lastly, the differences between using a discrete 5-point ACR scale and the extended continuous 7-point scales will be investigated. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

An Investigation Into the Impact of Test Session Disruptions for At-Home Test Administrations

In response to the closures of test centers worldwide due to the COVID-19 pandemic, several testing programs offered large-scale standardized assessments to examinees remotely. However, due to the varying quality of the performance of personal devices and internet connections, more at-home examinees likely suffered “disruptions” or an interruption in the connectivity to their testing session compared to typical test-center administrations. Disruptions have the potential to adversely affect examinees and lead to fairness or validity issues. The goal of this study was to investigate the extent to which disruptions impacted performance of at-home examinees using data from a large-scale admissions test. Specifically, the study involved comparing the average test scores of the disrupted examinees with those of the non-disrupted examinees after weighting the non-disrupted examinees to resemble the disrupted examinees along baseline characteristics. The results show that disruptions had a small negative impact on test scores on average. However, there was little difference in performance between the disrupted and non-disrupted examinees after removing records of the disrupted examinees who were unable to complete the test. [ FROM AUTHOR] Copyright of Applied Psychological Measurement is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full . (Copyright applies to all s.)

Interaural time difference based spatial release from masking with asymmetric hearing over a video conference app

Due to the difficulty of conducting offline laboratory experiments during the coronavirus outbreak, remote experiments such as experiments over video conference apps have become an important method to collect data for hearing researchers. For remote testing using headphone presentations, compared to monaural (i.e., unilateral or diotic) audio stimuli, dichotic stimuli (i.e., sounds with differences between left and right ears) are used in relatively less studies. In this study, a binaural hearing task of spatial release from masking (SRM) was tested in laboratory and over a video conference app, i.e., Tencent Meeting. In the experiment, the effects of interaural time difference (ITD) on SRM were compared between symmetric and asymmetric hearing which were realized by using a fixed interaural level difference (ILD) of 0 dB and -15 dB respectively. Results showed that 1) SRM was observed in the remote test but it was >4 dB smaller than the laboratory test;2) asymmetric hearing would lead to a ~2 dB significant decrease in the amount of masking release in both laboratory and remote conditions. The results indicate that binaural hearing could be measured remotely using the stereo sharing mode of video conference apps, but the effects of binaural cues especially ITDs may be degraded to some extent. © 2022 IEEE.

Cloud Follow-Up in Patients With Cardiovascular Implantable Electronic Devices: A Single-Region Study in China.

Background: Due to seriously imbalanced distribution of follow-up clinics in China, routine in-office visits are erratically attended by many cardiovascular implantable electronic device (CIED) patients. Meanwhile, remote monitoring is significantly underutilized. Novel tools to address the current predicament of routine in-office visits in China is urgently needed. Objectives: To assess the reliability and feasibility of cloud follow-up in CIED patients. Methods: A total of 325 CIED patients from 13 hospitals in Sichuan Province, China, were enrolled. Information on patients' sociodemographic and basic clinical characteristics was collected. All devices were tested and programmed with 5G-cloud follow-up platform in a real-time manner. All patients were surveyed about their acceptance of and preferences regarding cloud follow-up compared to routine in-office visits. Results: Compliance with routine in-office visits in this region was 60.6%. None of the patients were enrolled in remote monitoring services. Clinically important predictors of non-compliance were elderly age (≥75 years old), odds ratio (OR) 2.392 (95% confidence interval, 1.111-5.150); needing notification from a follow-up clinic, OR 2.518 (1.179-5.376); and being beyond 15 months post-implantation, OR 5.440 (2.563-11.543). All cloud follow-up sessions were performed safely and efficiently, without any adverse events. 292 (89.8%) patients preferred cloud follow-up for future device management. Conclusion: Compliance with routine in-office visits in this region has much room for improvement. Cloud follow-up addresses the limitations of an imbalanced distribution of follow-up clinics and geographic barriers for in-office CIED evaluation. Thus, cloud follow-up provides a potential solution to the current predicament of routine in-office visits in China.

IT-Supported Request Management for Clinical Radiology: Contextual Design and Remote Prototype Testing

Management of radiology requests in larger clinical contexts is characterized by a complex and distributed workflow. In our partner hospital, representing many similar clinics, these processes often still rely on exchanging physical papers and forms, making patient or case data challenging to access. This often leads to phone calls with long waiting queues, which are time-inefficient and result in frequent interrupts. We report on a user-centered design approach based on Rapid Contextual Design with an additional focus group to optimize and iteratively develop a new workflow. Participants found our prototypes fast and intuitive, the design clean and consistent, relevant information easy to access, and the request process fast and easy. Due to the COVID pandemic, we switched to remote prototype testing, which yielded equally good feedback and increased the participation rate. In the end, we propose best practices for remote prototype testing in hospitals with complex and distributed workflows. © 2022 Owner/Author.

Variations in consumer acceptance, sensory engagement and method practicality across three remote consumer-testing modalities.

In the context of the COVID-19 pandemic, it has become challenging for sensory scientists to conduct in-person sensory tests, particularly large central location tests. Sensory literature comparing central location and home use tests shows no clear consensus about how each methodology affects sample ratings and panelist engagement. Research on instructional delivery suggests that the most effective method of increasing engagement involves interactive video conferencing. The objective of this study was to compare three methods of remote consumer testing regarding sample acceptance, sensory engagement, and method practicality. Eighty-four participants rated five chocolate-chip cookie products on a 9-pt hedonic scale in each of three methods: 1) a live (synchronous) Zoom session, 2) an asynchronous video-guided session, and 3) a fully written protocol session. Results showed no significant differences in sample liking pattern across the methods used. Engagement scores approached the limit of significance for the Active Involvement dimension, indicating panelists were least likely to feel distracted, zoned out or lose interest in the written protocol method. There were no significant differences in the time spent on the test by the panelists across the three methods. Asynchronous methods showed to be most suitable in terms of the convenience of the time of day at which the tests were completed, but showed no significant differences in other aspects of method practicality. Overall, a written protocol method of remote consumer testing is recommended, as it is less time-consuming for researchers while providing similar acceptance and engagement as other methods.

Evaluating Spatial Hearing Using a Dual-Task Approach in a Virtual-Acoustics Environment.

Spatial hearing is critical for communication in everyday sound-rich environments. It is important to gain an understanding of how well users of bilateral hearing devices function in these conditions. The purpose of this work was to evaluate a Virtual Acoustics (VA) version of the Spatial Speech in Noise (SSiN) test, the SSiN-VA. This implementation uses relatively inexpensive equipment and can be performed outside the clinic, allowing for regular monitoring of spatial-hearing performance. The SSiN-VA simultaneously assesses speech discrimination and relative localization with changing source locations in the presence of noise. The use of simultaneous tasks increases the cognitive load to better represent the difficulties faced by listeners in noisy real-world environments. Current clinical assessments may require costly equipment which has a large footprint. Consequently, spatial-hearing assessments may not be conducted at all. Additionally, as patients take greater control of their healthcare outcomes and a greater number of clinical appointments are conducted remotely, outcome measures that allow patients to carry out assessments at home are becoming more relevant. The SSiN-VA was implemented using the 3D Tune-In Toolkit, simulating seven loudspeaker locations spaced at 30° intervals with azimuths between -90° and +90°, and rendered for headphone playback using the binaural spatialization technique. Twelve normal-hearing participants were assessed to evaluate if SSiN-VA produced patterns of responses for relative localization and speech discrimination as a function of azimuth similar to those previously obtained using loudspeaker arrays. Additionally, the effect of the signal-to-noise ratio (SNR), the direction of the shift from target to reference, and the target phonetic contrast on performance were investigated. SSiN-VA led to similar patterns of performance as a function of spatial location compared to loudspeaker setups for both relative localization and speech discrimination. Performance for relative localization was significantly better at the highest SNR than at the lowest SNR tested, and a target shift to the right was associated with an increased likelihood of a correct response. For word discrimination, there was an interaction between SNR and word group. Overall, these outcomes support the use of virtual audio for speech discrimination and relative localization testing in noise.

Remote Usability Testing Evaluation on the Most Visited E-commerce Website in Indonesia during Covid-19 Pandemic

During the Covid-19 pandemic, the online shopping platform help fulfill people's needs and it became a hub between the marketers to their customers. From the five biggest e-commerce services in Indonesia, Shopee is the most visited website by Indonesian citizens. The objectives of this study are to evaluate the effectiveness (E1) and efficiency (E2) of the website, and their correlation with website satisfaction rate based on usability variables. The synchronous remote testing technique was applied to 15 respondents to determine E1 and E2 levels. A usability rate perception was measured using a questionnaire to evaluate users' satisfaction (S), based on six usability variables (var 1-6). In this study, based on 18 tasks performed by 15 respondents, the effectiveness and efficiency rate of the website respectively 81%-98% and 73%-94%. The questionnaire result showed that the user satisfaction level of 313 respondents is 80%-84%. The Pearson correlation between E1, E2 with S respectively 0.619 and 0.814 (p-value<0.05). It concludes that Shopee is excellent e-commerce based on its effectiveness and efficiency, and user satisfaction (S) is mainly affected by its efficiency (E2). © 2021 IEEE.

Asynchronous Remote Assessment for Cognitive Impairment: Reliability Verification of the Neurotrack Cognitive Battery.

BACKGROUND: As evidenced by the further reduction in access to testing during the COVID-19 pandemic, there is an urgent, growing need for remote cognitive assessment for individuals with cognitive impairment. The Neurotrack Cognitive Battery (NCB), our response to this need, was evaluated for its temporal reliability and stability as part of ongoing validation testing. OBJECTIVE: The aim of this study is to assess the temporal reliability of the NCB tests (5 total) across a 1-week period and to determine the temporal stability of these measures across 3 consecutive administrations in a single day. METHODS: For test-retest reliability, a range of 29-66 cognitively healthy participants (ages 18-68 years) completed each cognitive assessment twice, 1 week apart. In a separate study, temporal stability was assessed using data collected from 31 different cognitively healthy participants at 3 consecutive timepoints in a single day. RESULTS: Correlations for the assessments were between 0.72 and 0.83, exceeding the standard acceptable threshold of 0.70 for temporal reliability. Intraclass correlations ranged from 0.60 to 0.84, indicating moderate to good temporal stability. CONCLUSIONS: These results highlight the NCB as a brief, easy-to-administer, and reliable assessment for remote cognitive testing. Additional validation research is underway to determine the full magnitude of the clinical utility of the NCB.

Unmoderated Remote Usability Testing: An Approach during Covid-19 Pandemic

Online Nurse Test for Indonesian Nurse Competency (ONT UKNI) is a mobile application that was developed to help increase the success rate of nurse competency test participants. By using this application, users can learn more about the materials tested and conduct try out as a competency test simulation. However, ONT UKNI has not yet passed adequate testing stages, especially in terms of User Interface/User Experience (UI/UX). The Covid-19 pandemic situation presents challenges in the UI/UX testing process. Testing process which is ideally carried out face-to-face with respondents to get further insight, have to be carried out using another approach following the new normal protocol. This study aims to test the usability of UI/UX with an unmoderated remote testing approach on ONT UKNI application using a USE questionnaire. The test was performed using 26 respondents and all were nursing profession students of Universitas Muhammadiyah Yogyakarta. Respondents performed 8 tasks on ONT UKNI and answered set of questionnaire that will be tabulated and analyzed. The results indicate that usefulness, ease of learning, and satisfaction variables get the Very Good category while the ease of use variable gets the Good category. Overall, usability testing using an unmoderated remote testing approach can be carried out and able to provide information about areas where users are satisfied with ONT UKNI application. However, some areas still have room for improvement such as better UI design and implementation of gamification © 2022, International Journal of Advanced Computer Science and Applications. All Rights Reserved.

Thinking outside the booth: when Covid-19 pushed sensory testing to remote options

This commentary addresses the issue of remote testing with sensory and consumer panels, within the VSI Covid-19 and Sensory Science in Food Quality and preference. In particular, two papers on the topic will be discussed: Dinnella et al. “Remote testing: Sensory test during Covid-19 pandemic and beyond” and Han Seok et al. “Stay safe in your vehicle: Drive-in booths can be an alternative to indoor booths for laboratory sensory testing”. At the time of writing this piece, the pandemic has been among us for two years, highlighting the fact that remote testing strategies are more needed than ever. Testing methods should be developed, compared, validated and optimized, so sensory practitioners are prepared and can have an informed methodological choice when their projects are pushed “outside the sensory booth”. © 2022 The Author(s)

Remote Dyslexia Screening for Bilingual Children

Ideally, language and reading skills in bilingual children are assessed in both languages spoken in order to avoid misdiagnoses of communication or learning disorders. Due to limited capacity of clinical and educational staff, computerized screenings that allow for automatic evaluation of the children’s performance on reading tasks (accuracy and speed) might pose a useful alternative in clinical and school settings. In this study, a novel web-based screening platform for language and reading assessment is presented. This tool has been preliminarily validated with monolingual Italian, Mandarin–Italian and English–Italian speaking primary school children living and schooled in Italy. Their performances in the screening tasks in Italian and—if bilingual—in their native language were compared to the results of standardized/conventional reading assessment tests as well as parental and teacher questionnaires. Correlations revealed the tasks that best contributed to the identification of risk for the presence of reading disorders and showed the general feasibility and usefulness of the computerized screening. In a further step, both screening administrators (Examiners) and child participants (Examinees) were invited to participate in usability studies, which revealed general satisfaction and provided suggestions for further improvement of the screening platform. Based on these findings, the potential of the novel web-based screening platform is discussed.